Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding of the performance trade-offs of different decisions made during the instruction-tuning process. These decisions include the scale and diversity of the instruction-tuning benchmark, different task sampling strategies, fine-tuning with and without demonstrations, training using specialized datasets for reasoning and dialogue, and finally, the fine-tuning objectives themselves. In this paper, we characterize the effect of instruction-tuning decisions on downstream task performance when scaling both model and benchmark sizes. To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks. Through the lens of this framework, we first present insights about instruction-tuning decisions as applied to OPT-30B and further exploit these insights to train OPT-IML 30B and 175B, which are instruction-tuned versions of OPT. OPT-IML demonstrates all three generalization abilities at both scales on four different evaluation benchmarks with diverse tasks and input formats -- PromptSource, FLAN, Super-NaturalInstructions, and UnifiedSKG. Not only does it significantly outperform OPT on all benchmarks but is also highly competitive with existing models fine-tuned on each specific benchmark. We release OPT-IML at both scales, together with the OPT-IML Bench evaluation framework.

Although weakly-supervised techniques can reduce the labeling effort, it is unclear whether a saliency model trained with weakly-supervised data (e.g., point annotation) can achieve the equivalent performance of its fully-supervised version. This paper attempts to answer this unexplored question by proving a hypothesis: there is a point-labeled dataset where saliency models trained on it can achieve equivalent performance when trained on the densely annotated dataset. To prove this conjecture, we proposed a novel yet effective adversarial trajectory-ensemble active learning (ATAL). Our contributions are three-fold: 1) Our proposed adversarial attack triggering uncertainty can conquer the overconfidence of existing active learning methods and accurately locate these uncertain pixels. {2)} Our proposed trajectory-ensemble uncertainty estimation method maintains the advantages of the ensemble networks while significantly reducing the computational cost. {3)} Our proposed relationship-aware diversity sampling algorithm can conquer oversampling while boosting performance. Experimental results show that our ATAL can find such a point-labeled dataset, where a saliency model trained on it obtained $97\%$ -- $99\%$ performance of its fully-supervised version with only ten annotated points per image.

Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users. While such a task enables many practical applications for interactive editing, existing methods often struggle to hallucinate realistic object instances in complex natural scenes. Such a limitation is partially due to the lack of semantic-level constraints inside the hole region as well as the lack of a mechanism to enforce realistic object generation. In this work, we propose a learning paradigm that consists of semantic discriminators and object-level discriminators for improving the generation of complex semantics and objects. Specifically, the semantic discriminators leverage pretrained visual features to improve the realism of the generated visual concepts. Moreover, the object-level discriminators take aligned instances as inputs to enforce the realism of individual objects. Our proposed scheme significantly improves the generation quality and achieves state-of-the-art results on various tasks, including segmentation-guided completion, edge-guided manipulation and panoptically-guided manipulation on Places2 datasets. Furthermore, our trained model is flexible and can support multiple editing use cases, such as object insertion, replacement, removal and standard inpainting. In particular, our trained model combined with a novel automatic image completion pipeline achieves state-of-the-art results on the standard inpainting task.

The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. To the best of our knowledge, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but not good enough for geoscience images by ignoring the geoscience related tasks. This paper aims to repair the occluded regions for a better geoscience task performance with the advanced visualization quality simultaneously, without changing the current deployed deep learning based geoscience models. Because of the complex context of geoscience images, we propose a coarse-to-fine encoder-decoder network with coarse-to-fine adversarial context discriminators to reconstruct the occluded image regions. Due to the limited data of geoscience images, we use a MaskMix based data augmentation method to exploit more information from limited geoscience image data. The experimental results on three public geoscience datasets for remote sensing scene recognition, cross-view geolocation and semantic segmentation tasks respectively show the effectiveness and accuracy of the proposed method.

作为最成功的AI驱动应用程序之一，推荐系统的目的是通过在我们生活的许多方面提供个性化建议，以有效而有效的方式帮助人们做出适当的决定，尤其是针对各种面向人类的在线服务，例如E-商务平台和社交媒体网站。在过去的几十年中，推荐系统的快速发展通过创造经济价值，节省时间和精力以及促进社会利益，从而使人类受益匪浅。但是，最近的研究发现，数据驱动的推荐系统可能会对用户和社会构成严重威胁，例如传播虚假新闻以操纵社交媒体网站中的公众舆论，扩大不公平为代表性不足的团体或在工作匹配服务中的个人，或从建议结果中推断隐私信息。因此，系统的可信赖性一直吸引着各个方面的关注，以减轻推荐系统引起的负面影响，以增强公众对推荐系统技术的信任。在这项调查中，我们提供了可信赖的推荐系统（TREC）的全面概述，特别关注六个最重要的方面；即安全与鲁棒性，非歧视与公平，解释性，隐私，环境福祉以及问责制和可审计性。对于每个方面，我们总结了最近的相关技术，并讨论了潜在的研究方向，以帮助未来实现值得信赖的推荐系统。

审议是人类日常生活中的一种共同自然行为。例如，在撰写论文或文章时，我们通常会首先编写草稿，然后迭代地擦亮它们，直到满足为止。鉴于这种人类的认知过程，我们提出了Decom，这是自动评论生成的多通审议框架。 DECOM由多个审议模型和一个评估模型组成。给定代码段，我们首先从代码中提取关键字，然后从预定义的语料库中检索类似的代码片段。然后，我们将检索到的代码的评论视为初始草案，并将其用代码和关键字输入到DETOM中，以开始迭代审议过程。在每次审议时，审议模型都会抛光草案并产生新的评论。评估模型衡量了新生成的评论的质量，以确定是否结束迭代过程。终止迭代过程后，将选择最佳的评论作为目标评论。我们的方法在Java（87K）和Python（108K）的两个现实世界数据集上进行了评估，实验结果表明，我们的方法表现优于最先进的基准。人类评估研究还证实，DECOM产生的评论往往更可读性，信息性和有用。

在过去的十年中，在线教育在为全球学生提供负担得起的高质量教育方面的重要性越来越重要。随着越来越多的学生改用在线学习，这在全球大流行期间得到了进一步放大。大多数在线教育任务，例如课程建议，锻炼建议或自动化评估，都取决于跟踪学生的知识进步。这被称为文献中的\ emph {知识跟踪}问题。解决此问题需要收集学生评估数据，以反映他们的知识演变。在本文中，我们提出了一个新的知识跟踪数据集，名为“知识跟踪数据库”练习（DBE-KT22），该练习是在澳大利亚澳大利亚国立大学教授的课程中从在线学生锻炼系统中收集的。我们讨论了DBE-KT22数据集的特征，并将其与知识追踪文献中的现有数据集进行对比。我们的数据集可通过澳大利亚数据存档平台公开访问。

顺序推荐通过历史互动来预测用户的下一个行为。推荐更长的序列可以提高建议准确性并提高个性化程度。随着序列的延长，现有作品尚未解决以下两个主要挑战。首先，在序列长度增加时，很难对远程内部序列依赖性进行建模。其次，它需要有效的内存和计算速度。在本文中，我们提出了一个稀疏的细心内存（SAM）网络，以进行长顺序用户行为建模。 SAM支持对用户行为序列的有效培训和实时推断，其长度为数千。在SAM中，我们将目标项目建模为查询和长序列作为知识数据库，在该数据库中，前者从后者中持续传达相关信息。 SAM同时模拟了目标序列依赖性和远程内部依赖性，其复杂性和O（1）顺序更新数量，只能通过具有O（l^2）复杂性的自我注意机制来实现这一目标。广泛的经验结果表明，我们提出的解决方案不仅在长期用户行为建模中而且在短序列建模中也有效。 SAM按照长度为1000的序列实施，成功部署在最大的国际电子商务平台之一上。此推论时间在30毫秒内，在线A/B测试的点击率提高了7.30％。据我们所知，这是第一个端到端的长用户序列建模框架，它以上述效率程度对序列和目标序列依赖性进行建模，并成功地部署在大型实时工业建议上系统。

深度神经网络极大地促进了单图超分辨率（SISR）的性能。传统方法仍然仅基于图像模态的输入来恢复单个高分辨率（HR）解决方案。但是，图像级信息不足以预测大型展望因素面临的足够细节和光真逼真的视觉质量（x8，x16）。在本文中，我们提出了一种新的视角，将SISR视为语义图像详细信息增强问题，以产生忠于地面真理的语义合理的HR图像。为了提高重建图像的语义精度和视觉质量，我们通过提出文本指导的超分辨率（TGSR）框架来探索SISR中的多模式融合学习，该框架可以从文本和图像模态中有效地利用信息。与现有方法不同，提出的TGSR可以生成通过粗到精细过程匹配文本描述的HR图像详细信息。广泛的实验和消融研究证明了TGSR的效果，该效果利用文本参考来恢复逼真的图像。

在许多应用程序问题（例如计算机视觉和自然语言处理）中，转移学习已成为一种强大的技术。但是，该技术在应用于遗传数据分析中被忽略了。在本文中，我们将转移学习技术与基于神经网络的方法（预期神经网络）相结合。通过转移学习，我们没有从头开始学习过程，而是从解决不同任务时学习的一项任务开始。我们利用先前的学习，并避免从头开始，以通过在不同但相关的任务中获得的信息来提高模型性能。为了演示性能，我们运行两个真实的数据集。通过使用转移学习算法，与预期神经网络相比，预期神经网络的性能得到了改善，而无需使用转移学习技术。